{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第一题：使用sklearn的多层感知机"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "实验内容：  \n",
    "1. 使用sklearn.neural_network.MLPClassifier完成手写数字分类任务\n",
    "2. 绘制学习率为3，1，0.1，0.01训练集损失函数的变化曲线"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 读取数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们使用的是sklearn里面自带的手写数字数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_digits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集有这几个键"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['data', 'target', 'frame', 'feature_names', 'target_names', 'images', 'DESCR'])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "load_digits().keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "打印数据集的描述"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _digits_dataset:\n",
      "\n",
      "Optical recognition of handwritten digits dataset\n",
      "--------------------------------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 5620\n",
      "    :Number of Attributes: 64\n",
      "    :Attribute Information: 8x8 image of integer pixels in the range 0..16.\n",
      "    :Missing Attribute Values: None\n",
      "    :Creator: E. Alpaydin (alpaydin '@' boun.edu.tr)\n",
      "    :Date: July; 1998\n",
      "\n",
      "This is a copy of the test set of the UCI ML hand-written digits datasets\n",
      "https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits\n",
      "\n",
      "The data set contains images of hand-written digits: 10 classes where\n",
      "each class refers to a digit.\n",
      "\n",
      "Preprocessing programs made available by NIST were used to extract\n",
      "normalized bitmaps of handwritten digits from a preprinted form. From a\n",
      "total of 43 people, 30 contributed to the training set and different 13\n",
      "to the test set. 32x32 bitmaps are divided into nonoverlapping blocks of\n",
      "4x4 and the number of on pixels are counted in each block. This generates\n",
      "an input matrix of 8x8 where each element is an integer in the range\n",
      "0..16. This reduces dimensionality and gives invariance to small\n",
      "distortions.\n",
      "\n",
      "For info on NIST preprocessing routines, see M. D. Garris, J. L. Blue, G.\n",
      "T. Candela, D. L. Dimmick, J. Geist, P. J. Grother, S. A. Janet, and C.\n",
      "L. Wilson, NIST Form-Based Handprint Recognition System, NISTIR 5469,\n",
      "1994.\n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "  - C. Kaynak (1995) Methods of Combining Multiple Classifiers and Their\n",
      "    Applications to Handwritten Digit Recognition, MSc Thesis, Institute of\n",
      "    Graduate Studies in Science and Engineering, Bogazici University.\n",
      "  - E. Alpaydin, C. Kaynak (1998) Cascading Classifiers, Kybernetika.\n",
      "  - Ken Tang and Ponnuthurai N. Suganthan and Xi Yao and A. Kai Qin.\n",
      "    Linear dimensionalityreduction using relevance weighted LDA. School of\n",
      "    Electrical and Electronic Engineering Nanyang Technological University.\n",
      "    2005.\n",
      "  - Claudio Gentile. A New Approximate Maximal Margin Classification\n",
      "    Algorithm. NIPS. 2000.\n"
     ]
    }
   ],
   "source": [
    "print(load_digits()['DESCR'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一共1797个样本，每个样本都是$8 \\times 8$的矩阵，因为是灰度的图像，所以只有一个通道  \n",
    "images对应的是原始的图像，data对应的是 $8 \\times 8$ reshape成 $1 \\times 64$ 的数据，target是标记，表示这张图片里面的数字是几"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "打印第一个样本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  5., 13.,  9.,  1.,  0.,  0.],\n",
       "       [ 0.,  0., 13., 15., 10., 15.,  5.,  0.],\n",
       "       [ 0.,  3., 15.,  2.,  0., 11.,  8.,  0.],\n",
       "       [ 0.,  4., 12.,  0.,  0.,  8.,  8.,  0.],\n",
       "       [ 0.,  5.,  8.,  0.,  0.,  9.,  8.,  0.],\n",
       "       [ 0.,  4., 11.,  0.,  1., 12.,  7.,  0.],\n",
       "       [ 0.,  2., 14.,  5., 10., 12.,  0.,  0.],\n",
       "       [ 0.,  0.,  6., 13., 10.,  0.,  0.,  0.]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "load_digits()['images'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对数据集的前十张图片可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj8AAABRCAYAAAA9zcc0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAANY0lEQVR4nO3df+xVdR3H8ddbcImi/Ch/FBmISrlaXxz8VTNgQWatoAxnmKJbi+l00rTJHzpFbcKWofNXsBlQtjbZDMpcThNYuvUDQnOKMX+RmgzRr4CiWPDuj3PZmH3e6Plyzvlc7nk+tjvxvcu973Pv55z75pzP+3zM3QUAANAWh+VOAAAAoEkUPwAAoFUofgAAQKtQ/AAAgFah+AEAAK1C8QMAAFqF4gcAALRK9uLHzEaa2W/M7G0z22xms3LnVDUzu9TM1pnZbjNbljufqpnZR8zs7s73t9PMNpjZWbnzqpqZ3WNmr5rZDjPbZGbfz51THczsVDN718zuyZ1L1cxsTWfb3uo8/pk7pzqY2blmtrFzXH3OzM7InVNV9vvu9j32mNltufOqkpmNMbMHzKzfzLaY2e1mNjh3XlUys9PM7BEz225mz5rZt5p8/+zFj6Q7JL0n6XhJ50m6y8w+mzelyv1b0o2Sfp47kZoMlvSSpEmShkm6RtK9ZjYmY051uEnSGHc/RtI3Jd1oZhMy51SHOyT9LXcSNbrU3Yd2Hp/OnUzVzGyapIWSLpJ0tKQvSXo+a1IV2u+7G6rid+MdSSsyp1W1OyVtlfRxSeNVHFsvyZpRhTqF3CpJ90saKekHku4xs3FN5ZC1+DGzoySdLekad3/L3R+V9FtJ5+fMq2rufp+7r5T0eu5c6uDub7v7de7+orvvdff7Jb0gqacKA3d/yt137/vfzuPkjClVzszOlfSmpD/mzgUDNl/S9e7+587++Iq7v5I7qZp8R0WR8KfciVTsJEn3uvu77r5F0h8k9dJJgc9I+oSkRe6+x90fkfSYGvztz33mZ5ykPe6+ab/YE+qtL7l1zOx4Fd/tU7lzqZqZ3WlmuyQ9I+lVSQ9kTqkyZnaMpOslXZE7l5rdZGbbzOwxM5ucO5kqmdkgSRMlHdu5lPBy55LJkNy51WS2pF94763TdKukc83sSDMbJeksFQVQr7Ag9rmmEshd/AyVtP19se0qTtXiEGRmh0v6laTl7v5M7nyq5u6XqBifZ0i6T9LuA/+NQ8oNku5295dyJ1KjqySNlTRK0hJJvzOzXjp7d7ykw1WcETlDxSWT0yVdnTOpOpjZp1RcDlqeO5carFVxEmCHpJclrZO0MmtG1XpGxRm7H5nZ4Wb2FRXf5ZFNJZC7+HlL0jHvix0jaWeGXHCQzOwwSb9UMYfr0szp1KZzmvZRSZ+UdHHufKpgZuMlTZW0KHcudXL3v7j7Tnff7e7LVZxq/1ruvCr0Tue/t7n7q+6+TdJP1VvbuM8Fkh519xdyJ1KlznH0QRX/uDpK0sckjVAxj6snuPt/JM2Q9HVJW1Scbb5XRaHXiNzFzyZJg83s1P1iferByyW9zsxM0t0q/uV5dmdw97rB6p05P5MljZH0LzPbIulKSWeb2d9zJtUAV/oU/CHJ3ftV/ID02mWglAvUm2d9Rko6UdLtnSL9dUlL1WMFrLv/w90nuftH3f1MFWdk/9rU+2ctftz9bRXV7fVmdpSZfVHSdBVnD3qGmQ02syMkDZI0yMyO6LW2RUl3STpN0jfc/Z0PevKhxsyO67QPDzWzQWZ2pqTvSnokd24VWaKikBvfefxM0u8lnZkzqSqZ2XAzO3Pf/mdm56nohHowd24VWyrpss6YHSFproqump5hZl9Qcemy17q81Dlb94KkizvjdLiKuU1P5M2sWmb2+c6+eKSZXamis21ZU++f+8yPVLTvDVFx/e/Xki52914783O1itPR8yR9r/PnnrkGb2ajJc1R8aO5Zb/7b5yXObUquYpLXC9L6pf0E0lz3X1V1qwq4u673H3LvoeKS9LvuvtruXOr0OEqbjnxmqRtki6TNMPde+1ePzeouFXBJkkbJW2Q9OOsGVVvtqT73L1Xp0h8W9JXVYzVZyX9V9IPs2ZUvfNVNI1slfRlSdP266atnfXeJHkAAIBYN5z5AQAAaAzFDwAAaBWKHwAA0CoUPwAAoFUofgAAQKuUuteMmZVqDRsxYkQyPmrUqGR8x44dyfgrr6TX5NuzZ0+ZdOTuB7yZWdnti4wbl16YdvDg9Mcdbd/27e9f+eMDbXP3Yw/0hKq2cejQocn4Kaeckozv2rUrGd+0aVMyHvmg71Aqv40nnHBCMh6N0927092YGzduTMarHqdSdd/joEGDkvExY8Yk488991wVb1v5vhjtc++9914y/uKLL5Z5+YFobF8se7x5+umnq3jbWsbpcccdl4xH4zT6jRkyJL2UWbQvPvnkk/8X27t3r/bu3Vv5Np544onJ+PDhw5Pxbdu2JeNbt25NxnP/Lp58cvq+r9F3WPY3YACS+2KtN9qbOnVqMr5gwYJk/OGHH07G582bl4z39/cPLLGaLVmyJBmPBve1116bjK9aVfoWMpvL/oWBmjhxYjK+cmV6+ZnHH388GZ88eXJVKQ3Y7Nmzk/FonD7//PPJePSZdOs4laSjj04vo3fzzTcn4zNmzKgznQGL9rmoyLnwwgvrS6bQ2L5Y9ngzfvz4OtM5KLNmzUrGo22JxmNfX18yHv2DMlXs79xZzy2ErrgivW5wtC3Lli1Lxm+55ZZk/M033xxQXlWJjh3Rd9jAb0ByX+SyFwAAaBWKHwAA0CoUPwAAoFVqnfMTzZkYO3ZsMh5NXnvjjTeS8XPOOScZX7Ei71p30TXXSZMmJeNTpkxJxgcw56dy0fyA1atXJ+Nlrqk3LRqPM2fOTMbnzJmTjC9evDgZnzBhQjIezWXrBtHcl2iOVreKxle0z0XzvDZvTk/V6YbxO3369GQ82sb58+fXmU6jomPq3LlzS8WjeSdNzpMpO+cq2kejuTJNzaOM9olonEaiJbaeeCK9jmtVc9Y48wMAAFqF4gcAALQKxQ8AAGgVih8AANAqFD8AAKBVKun2irpcoq6u6PbX0Z1zH3rooVLv21S3VzTrvOxs+27urInuOhrNxI/u8BzdxbpJ0Z1wFy5cmIyvW7cuGY/GaTd3dUVdLlEnSXT32LJdTw0sIyEp7tYZPXp0Mh51Ja5ZsyYZ74YuobLdW9G+2M2icRe57rrrkvFonHbDHeWj433Zu5FHYy/axmhsD1S0T0TWrl2bjEfbXfd3xZkfAADQKhQ/AACgVSh+AABAq1D8AACAVqH4AQAArVJJt1e0Jtf69euT8ahbJhK9TlOidWKiToNhw4aVev2qZ+FXKeq+iGboR8/vhnXKonEXdSVG8airK9oP+vv7P0R29Yo6RqKumGXLliXj0fcbdZ5E+0jVovHY19eXjEf7aNSJ02RXVyTqrok6L7u5i7SqdamiY3Mk6l6NxnsdovfasGFDMh7to9GYbKrDsuz7RJ991JVYtpusLM78AACAVqH4AQAArULxAwAAWoXiBwAAtArFDwAAaJVau72qWusodxdN1OESzdovm1fds9oPJoeomyKauR+Juo26QdQFNnLkyGQ8Wmsuik+bNi0Zr2P8Tp8+PRlftGhRMr58+fJSr3/55Zcn4xdddFGp16laNB6j7qFoXb7oc4qUXYvqYET7aNR1E+27UXdNU11CB3qvqtZLjMZDN3TWlj3eT5o0KRk/6aSTkvHc6+lF3YfR8e7WW29NxqOxEHW/ld1uzvwAAIBWofgBAACtQvEDAABaheIHAAC0CsUPAABolUq6vaJZ3BMmTCj1OlFXV/Q6K1asKPX63Sqa1d7k2jzRGkxRd08k6rLohrWRyorGddS9tXjx4mT8qquuSsbnzZs3sMQOYPv27aXis2fPTsajMRmJOohyq6q7J+owaVLUzRJ1A0VdRVFH2+mnn56M13EcirYlOn64e6nnd0NXV7QPrV69OhmfP39+Mh6NvWifiz6TprrAou2u6ncu6rAs24HMmR8AANAqFD8AAKBVKH4AAECrUPwAAIBWofgBAACtUkm3V7Q2UtSlNXPmzFLxyMKFC0s9H7FonbJoTZ2+vr5kPOpAWLVqVTK+dOnSUs+vw4IFC5LxaG26qCtx6tSpyXiTXYlRl0vU+RN1YESvE60FlrubL1rTLOpyi7obI93QzRbto1H3VtTdE3UPRd0yTXadRp080fe4du3aOtM5KNHnH21LtO3R97Vhw4ZkPFpHseyYr1o0jqLtjrajbFdXhDM/AACgVSh+AABAq1D8AACAVqH4AQAArULxAwAAWqXWbq9o7aKou2b9+vXJ+MSJEweWWM2iDpeoUynqSIk6qqLujjpEM/HLrtMSdRRE2x51RDTZ7RWt4RWt1RWJurrmzJlTOqemRGN42LBhyXiTY7KMKVOmJONl16aLutm6Ya2o6LOPuoGibploW7qhoy06FkZr0OXuMjyQKLfo84+OQ1F3WHSMjLqnmhK9f/SbEXWiRmOhqu5DzvwAAIBWofgBAACtQvEDAABaheIHAAC0CsUPAABoFXP3D/9ks9ckba4vnVqNdvdjD/SEQ3z7pN7fxg/cPoltPAT0+jiVen8bGacdvb6Nh/j2ScE2lip+AAAADnVc9gIAAK1C8QMAAFqF4gcAALQKxQ8AAGgVih8AANAqFD8AAKBVKH4AAECrUPwAAIBWofgBAACt8j8+0X8dWT+/EAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x288 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, figs = plt.subplots(1, 10, figsize=(10, 4))\n",
    "for f, img, lbl in zip(figs, load_digits()['images'][:10], load_digits()['target'][:10]):\n",
    "    f.imshow(img, cmap = 'gray')\n",
    "    f.set_title(lbl)\n",
    "    f.axes.get_xaxis().set_visible(False)\n",
    "    f.axes.get_yaxis().set_visible(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 划分数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "取40%为测试集，60%为训练集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainX, testX, trainY, testY = train_test_split(load_digits()['data'], load_digits()['target'], test_size = 0.4, random_state = 32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1078, 64), (1078,), (719, 64), (719,))"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainX.shape, trainY.shape, testX.shape, testY.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "神经网络的训练方法一般是基于梯度的优化算法，如梯度下降，为了让这类算法能更好的优化神经网络，我们往往需要对数据集进行归一化，这里我们选择对数据进行标准化\n",
    "\n",
    "$$X' = \\frac{X - \\bar{X}}{\\mathrm{std}(X)}$$\n",
    "\n",
    "其中，$\\bar{X}$是均值，$\\mathrm{std}$是标准差。减去均值可以让数据以0为中心，除以标准差可以让数据缩放到一个较小的范围内。这样可以使得梯度的下降方向更多样，同时缩小梯度的数量级，让学习变得稳定。  \n",
    "首先需要对训练集进行标准化，针对每个特征求出其均值和标准差，然后用训练集的每个样本减去均值除以标准差，就得到了新的训练集。然后用测试集的每个样本，减去训练集的均值，除以训练集的标准差，完成对测试集的标准化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 初始化一个标准化器的实例\n",
    "standard = StandardScaler()\n",
    "\n",
    "# 对训练集进行标准化，它会计算训练集的均值和标准差保存起来\n",
    "trainX = standard.fit_transform(trainX)\n",
    "\n",
    "# 使用标准化器在训练集上的均值和标准差，对测试集进行归一化\n",
    "testX = standard.transform(testX)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以打印看一下数据集归一化后的效果，均值很接近0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1.4418480839287748e-18, -0.005673919242693978)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainX.mean(), testX.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 引入模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neural_network import MLPClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们使用sklearn中自带的MLPClassifier，MLP是多层感知机(multi-layer perceptron)的简称。  \n",
    "在训练的时候需要指定参数，这里我们需要设置的几个参数有：\n",
    "1. solver: 'sgd'，这个参数的含义是，使用随机梯度下降作为优化算法\n",
    "2. learning_rate: 'constant'，学习率固定，不衰减\n",
    "3. momentum: 0，动量设置为0，这是随机梯度下降需要的一个参数，我们设置为0即可\n",
    "4. max_iter: 设定最大迭代轮数，如果超过这个轮数还没有收敛，就停止训练，并抛出一个warning\n",
    "5. learning_rate_init，这个参数需要我们进行调整，这是学习率"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个模型会判断，如果连续两轮损失值都没有减少了，就停止训练。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = MLPClassifier(solver = 'sgd', learning_rate = 'constant', momentum = 0, learning_rate_init = 0.1, max_iter = 500)\n",
    "model.fit(trainX, trainY)\n",
    "prediction = model.predict(testX)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. 预测与评估"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9833101529902643"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_score(prediction, testY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "精度达到了98%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. 绘制训练集损失函数值的变化曲线"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以通过model.loss_curve_获取模型在训练过程中，损失函数损失值的变化曲线"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'loss')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAFzCAYAAAB2A95GAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3Scd33n8c93npmRNLJsSbacOL7ESRxy4RKSuiEJDU1LoUlKG9pCG0ohpHSzdEMPbPd0aZde9vTs6dILPUtLS8gChbTZpAWakqahUFhIYENIbGOHJM7FDiG+xZbvliVLc/nuH88z8kie0XV+eqTR+3WOzsz85nlmvo8eyfr49/vN7zF3FwAAAOZWJu0CAAAAFiNCGAAAQAoIYQAAACkghAEAAKSAEAYAAJACQhgAAEAKsmkXMF0rVqzw9evXp10GAADApDZv3nzQ3fvqPbfgQtj69eu1adOmtMsAAACYlJn9sNFzDEcCAACkgBAGAACQAkIYAABACghhAAAAKSCEAQAApIAQBgAAkAJCGAAAQAoIYQAAACkghAEAAKSAEAYAAJACQhgAAEAKCGHjnDhV1DeePaD+E8NplwIAAFoYIWyclw4P6ta/fVxbXjqSdikAAKCFEcLGyUfxt6RYrqRcCQAAaGWEsHFyhDAAADAHCGHj5LJJCCt5ypUAAIBWRggbJxeZJGmEnjAAABAQIWycXIbhSAAAEB4hbJzR4UhCGAAACIgQNk51OLJYZk4YAAAIhxA2DsORAABgLhDCxslkTNmMEcIAAEBQhLA6clGG4UgAABAUIayOXGQaKdETBgAAwiGE1ZHPZhiOBAAAQRHC6oiHIwlhAAAgHEJYHcwJAwAAoQULYWa21sy+YWbbzewpM/tAnW3MzP7SzHaY2RNmdkWoeqYjGxmXLQIAAEFlA752SdJ/cfctZtYlabOZ/bu7P12zzQ2SLky+XifpE8ltqvJRRkUm5gMAgICC9YS5+z5335LcPyFpu6TV4za7SdJdHntUUreZrQpV01QxJwwAAIQ2J3PCzGy9pMslfXfcU6sl7ap5vFtnBjWZ2W1mtsnMNvX394cqc1QuMuaEAQCAoIKHMDNbIumLkj7o7sfHP11nlzPSj7vf6e4b3X1jX19fiDLHoCcMAACEFjSEmVlOcQC7293/qc4muyWtrXm8RtLekDVNBeuEAQCA0EJ+OtIkfVrSdnf/iwab3S/p3cmnJK+SdMzd94WqaapYogIAAIQW8tORr5f0LknfN7OtSdt/k7ROktz9DkkPSrpR0g5Jg5JuDVjPlMVzwugJAwAA4QQLYe7+bdWf81W7jUu6PVQNM5WLMqwTBgAAgmLF/DryTMwHAACBEcLqyEUZFUvMCQMAAOEQwurIMicMAAAERgirgzlhAAAgNEJYHawTBgAAQiOE1ZGLTCXWCQMAAAERwurIRRmVKq5KhSAGAADCIITVkYvib0uxwpAkAAAIgxBWR74awhiSBAAAgRDC6shF8UL/xRI9YQAAIAxCWB25bLUnjBAGAADCIITVUZ0TxlphAAAgFEJYHaPDkcwJAwAAgRDC6hj9dCQ9YQAAIBBCWB2jw5FMzAcAAIEQwuqoLlFRYrFWAAAQCCGsDoYjAQBAaISwOlgnDAAAhEYIq6O6ThhLVAAAgFAIYXVw2SIAABAaIawO5oQBAIDQCGF1nF6slRAGAADCIITVwTphAAAgNEJYHTnmhAEAgMAIYXUwHAkAAEIjhNVRXaKCEAYAAEIhhNXBEhUAACA0QlgdLFEBAABCI4TVEWVMGSOEAQCAcAhhDeSiDJctAgAAwRDCGshHGRVLzAkDAABhEMIayGUzDEcCAIBgCGENZDNGCAMAAMEQwhpgThgAAAiJENZAPpthnTAAABAMIayBXGQqcgFvAAAQCCGsgVyUUalCCAMAAGEQwhqI54QxHAkAAMIghDUQrxNGTxgAAAiDENZALssSFQAAIBxCWAO5iMVaAQBAOISwBpgTBgAAQiKENZCLGI4EAADhEMIaYDgSAACERAhrIMenIwEAQECEsAaYEwYAAEIihDWQj4wV8wEAQDCEsAYYjgQAACERwhrIZTMqMhwJAAACIYQ1EM8Jq8idIAYAAJqPENZAPjJJUqlCCAMAAM1HCGsgF8XfGtYKAwAAIRDCGshWQ1iJnjAAANB8hLAGqsORI/SEAQCAAAhhDTAcCQAAQiKENUAIAwAAIRHCGshlqyGMOWEAAKD5CGENVOeE0RMGAABCIIQ1wHAkAAAIiRDWACEMAACERAhroBrCRlgnDAAABBAshJnZZ8zsgJk92eD568zsmJltTb7+IFQtM5HPMicMAACEkw342p+V9HFJd02wzbfc/S0Ba5gxhiMBAEBIwXrC3P1hSYdDvX5o2QwhDAAAhJP2nLCrzWybmX3ZzF7ZaCMzu83MNpnZpv7+/jkprDocOcI6YQAAIIA0Q9gWSee6+2WS/krSPzfa0N3vdPeN7r6xr69vToobHY4s0RMGAACaL7UQ5u7H3X0guf+gpJyZrUirnvGYEwYAAEJKLYSZ2dlmZsn9K5NaDqVVz3ijIazCcCQAAGi+YJ+ONLN7JF0naYWZ7Zb0h5JykuTud0h6m6TfMLOSpCFJN7v7vEk8eYYjAQBAQMFCmLu/Y5LnP654CYt5Kcc6YQAAIKC0Px05bzEnDAAAhEQIayCbYYkKAAAQDiGsATNTPsrQEwYAAIIghE0gGxkT8wEAQBCEsAnk6AkDAACBEMImkIsyzAkDAABBEMImkI+MnjAAABAEIWwCuWxGJUIYAAAIgBA2gXhOGMORAACg+QhhE4jnhNETBgAAmo8QNgHmhAEAgFAIYRNgiQoAABAKIWwCuSijYok5YQAAoPkIYRPIRsacMAAAEAQhbAL5KKMRLlsEAAACIIRNoD0f6VSpnHYZAACgBRHCJtCZjzQ4TAgDAADNRwibQCGf1cmRUtplAACAFkQIm0BnW6TBkbLc+YQkAABoLkLYBAr5rMoV1zCT8wEAQJMRwibQmY8kSUMjzAsDAADNRQibQKEtK0nMCwMAAE1HCJtAZz4OYYP0hAEAgCYjhE2g0BYPR54cpicMAAA0FyFsAvSEAQCAUAhhEyjk6QkDAABhEMIm0NlGTxgAAAiDEDaB0Z4wPh0JAACajBA2gWoI4/qRAACg2QhhEyjkWScMAACEQQibQJQxtecyzAkDAABNRwibRGc+y6cjAQBA0xHCJlFoi+gJAwAATUcImwQ9YQAAIARC2CQKeXrCAABA8xHCJtHZltUgn44EAABNRgibBD1hAAAgBELYJDrzWdYJAwAATUcIm0ShLWLFfAAA0HSEsEnQEwYAAEIghE2ikM/qVLGicsXTLgUAALQQQtgkOtuSi3jTGwYAAJqIEDaJ6kW8+YQkAABoJkLYJKo9YayaDwAAmokQNomOXHU4kp4wAADQPISwSXS2xcOR9IQBAIBmIoRNopCnJwwAADQfIWwSoz1hfDoSAAA00ZRCmJl9wMyWWuzTZrbFzN4curj5YLQnjFXzAQBAE021J+zX3P24pDdL6pN0q6SPBKtqHunM0xMGAACab6ohzJLbGyX9rbtvq2lraYU25oQBAIDmm2oI22xmX1Ucwr5iZl2SKuHKmj/yUUbZjLFiPgAAaKrsFLd7r6TXSnrB3QfNrFfxkGTLMzMV8pFOMicMAAA00VR7wq6W9Ky7HzWzX5X0e5KOhStrfulsy9ITBgAAmmqqIewTkgbN7DJJ/1XSDyXdFayqeaaQj3SSOWEAAKCJphrCSu7ukm6S9DF3/5ikrnBlzS+dbVkNsmI+AABooqnOCTthZr8r6V2SrjWzSFIuXFnzCz1hAACg2abaE/bLkoYVrxf2sqTVkv4sWFXzTGeeOWEAAKC5phTCkuB1t6RlZvYWSafcffHMCWvLsmI+AABoqqletuiXJD0m6e2SfknSd83sbSELm0868xEr5gMAgKaa6pywD0v6UXc/IElm1ifpa5K+EKqw+aSQpycMAAA011TnhGWqASxxaLJ9zewzZnbAzJ5s8LyZ2V+a2Q4ze8LMrphiLXOukPSExR8QBQAAmL2phrB/M7OvmNl7zOw9kv5V0oOT7PNZSddP8PwNki5Mvm5TvBbZvFRoi1Rxabi0KK7UBAAA5sCUhiPd/bfN7BclvV7xhbvvdPf7JtnnYTNbP8EmN0m6K1l/7FEz6zazVe6+b2qlz53OfPxtOjlcUnsuSrkaAADQCqY6J0zu/kVJX2zie6+WtKvm8e6k7YwQZma3Ke4t07p165pYwtQU8nHwGhwpa/mcvzsAAGhFE4YwMzshqd5EKJPk7r50Fu9tddrqTrpy9zsl3SlJGzdunPOJWZ1t8bdpkAVbAQBAk0wYwtw95KWJdktaW/N4jaS9Ad9vxqo9YSxTAQAAmmWqE/NDuF/Su5NPSV4l6dh8nA8m1fSEsUwFAABokinPCZsuM7tH0nWSVpjZbkl/qOR6k+5+h+JPV94oaYekQUm3hqpltqoT8weGiylXAgAAWkWwEObu75jkeZd0e6j3b6bezrwk6fBJQhgAAGiONIcjF4zuQk6SdGRwJOVKAABAqyCETUF7LlJnPtLhk4QwAADQHISwKerpzOsIIQwAADQJIWyKejvzOsxwJAAAaBJC2BT1FOgJAwAAzUMImyJ6wgAAQDMRwqYo7gljiQoAANAchLAp6u3MaWC4pOESq+YDAIDZI4RNUU+yYOvRQXrDAADA7BHCpqi3UF01n3lhAABg9ghhU1TtCeMTkgAAoBkIYVO0PAlhhwhhAACgCQhhUzTaE8YyFQAAoAkIYVPU3RFfxJs5YQAAoBkIYVOUjTJa1pFjThgAAGgKQtg0xKvms0QFAACYPULYNPQU6AkDAADNQQibht7OPHPCAABAUxDCpqGnkOfTkQAAoCkIYdNQ7Qlz97RLAQAACxwhbBp6OvMaLlU0VOQi3gAAYHYIYdPA9SMBAECzEMKmobpqPiEMAADMFiFsGnoJYQAAoEkIYdPQy/UjAQBAkxDCpuH0nDBWzQcAALNDCJuGrvasooyxaj4AAJg1Qtg0ZDKmnkJOhxmOBAAAs0QIm6aeQp6eMAAAMGuEsGnq4fqRAACgCQhh09S3pE39J4bTLgMAACxwhLBpWtPTod1Hh1SpcP1IAAAwc4SwaVrT06GRUkX9A/SGAQCAmSOETdOa3oIkafeRwZQrAQAACxkhbJrW9nRIknYdHkq5EgAAsJARwqZpTU/cE7brMD1hAABg5ghh09Sei7RiSZt2H6EnDAAAzBwhbAbW9nZoF3PCAADALBDCZmBtT4GeMAAAMCuEsBlY09OhvUeHVGatMAAAMEOEsBlY21tQqeLad4zeMAAAMDOEsBlYkyxTwZAkAACYKULYDKxlmQoAADBLhLAZWNXdLjN6wgAAwMwRwmagLRvp7KXtLFMBAABmjBA2Q2t7CtrNpYsAAMAMEcJmaE1PBxfxBgAAM0YIm6E1vQXtO35KI6VK2qUAAIAFiBA2Q2t6OuQu1goDAAAzQgiboeoyFS+xTAUAAJgBQtgMXbCyU5L0/P6BlCsBAAALESFshlZ2tWvFkrye3nc87VIAAMACRAibhUtWLdV2QhgAAJgBQtgsXLpqqZ7fP6BimU9IAgCA6SGEzcKl5yzVSLminf3MCwMAANNDCJuFS1YtlSQ9vZchSQAAMD2EsFk4f0Wn8tkM88IAAMC0EcJmIRtldNFZXdq+70TapQAAgAWGEDZLl65aqqf3HZe7p10KAABYQAhhs3TJqi4dPjmiAyeG0y4FAAAsIISwWbr0nGWSmJwPAACmJ2gIM7PrzexZM9thZr9T5/nrzOyYmW1Nvv4gZD0hXLyqS5JYOR8AAExLNtQLm1kk6a8lvUnSbkmPm9n97v70uE2/5e5vCVVHaEvbc1rb20EIAwAA0xKyJ+xKSTvc/QV3H5F0r6SbAr5fal65apm27TqadhkAAGABCRnCVkvaVfN4d9I23tVmts3Mvmxmr6z3QmZ2m5ltMrNN/f39IWqdlavO79XuI0PadXgw7VIAAMACETKEWZ228es4bJF0rrtfJumvJP1zvRdy9zvdfaO7b+zr62tymbN3zYYVkqTv7DyUciUAAGChCBnCdktaW/N4jaS9tRu4+3F3H0juPygpZ2YrAtYUxIUrl2jFkjY9svNg2qUAAIAFImQIe1zShWZ2npnlJd0s6f7aDczsbDOz5P6VST0LrjvJzHTNBcv1yM5DLNoKAACmJFgIc/eSpPdL+oqk7ZL+0d2fMrP3mdn7ks3eJulJM9sm6S8l3ewLNMVcc8FyHTgxrJ39A2mXAgAAFoBgS1RIo0OMD45ru6Pm/sclfTxkDXPlmgviUdRHdh7ShpVdKVcDAADmO1bMb5K1vR1a3d2hR3YsuNFUAACQAkJYk5iZXr9hub7zwiFVKgtyRBUAAMwhQlgTXXPBCh0bKuopriMJAAAmQQhroh+7cIUyJn316ZfTLgUAAMxzhLAmWrGkTVdfsFwPPLGPpSoAAMCECGFN9rOvOUc/OHiSIUkAADAhQliTXf+qs5XNmP5l297JNwYAAIsWIazJugt5XXvhCoYkAQDAhAhhAfzsZedoz9EhbXnpaNqlAACAeYoQFsCbLj1L+WyGIUkAANAQISyArvacfuqSlfrS1j0aGimnXQ4AAJiHCGGB3HL1eh0ZLOq+7+1JuxQAADAPEcICufK8Xr169TJ9+tsvcBkjAABwBkJYIGamX7/2PO3sP6mHnutPuxwAADDPEMICuvHVq3T20nZ96tsvpF0KAACYZwhhAeWijG65Zr3+345DenLPsbTLAQAA8wghLLBfed06LevI6U/+7Zm0SwEAAPMIISywZR05/eZPbtC3nj/I3DAAADCKEDYH3n31ep27vKA//tftKvNJSQAAIELYnMhnM/rQ9Rfr2f0n9PlNu9IuBwAAzAOEsDlyw6vO1o+c26M//cqzOjgwnHY5AAAgZYSwOWJm+uOff7UGTpX0+//8pNwZlgQAYDEjhM2hi87u0gffdKG+/OTLeuCJfWmXAwAAUkQIm2O3XXu+Xru2W7//pSd14PiptMsBAAApIYTNsWyU0Z+//TKdKpb1/nu+p2K5knZJAAAgBYSwFGxYuUQf+YXX6LEfHNaffJlFXAEAWIwIYSl56+WrdcvV5+pT3/6BHnhib9rlAACAOUYIS9GHf+ZSXbGuW7/1j9v0yI6DaZcDAADmECEsRflsRp+65Ue1fnlBv37XJm3+4eG0SwIAAHOEEJay3s68/v69r9PKrja9528f1/deOpJ2SQAAYA4QwuaBlUvbdfd/uEo9hbze+anv6mEu9A0AQMsjhM0Tq7s79IXfuFrnLu/Uez/3uL60dU/aJQEAgIAIYfPIyq523XvbVbp8XY8+cO9W/c8Ht6vEOmIAALQkQtg8s6wjp79775X61avW6ZMPv6B3ffoxVtYHAKAFEcLmobZspP/x1lfrz99+mba8dERv/l8P61+2sZYYAACthBA2j73tR9bowQ9cq3OXd+o37/mebr97C71iAAC0CELYPHdB3xJ98X1X67d/+iL9+/b9euNHH9Jd33lR5YqnXRoAAJgFQtgCkI0yuv0nNugrH3yDLlvbrT/40lO64WMP6xvPHJA7YQwAgIWIELaAnLeiU3/33iv1iXdeoZFSRbd+9nH98icf1cPP9RPGAABYYGyh/fHeuHGjb9q0Ke0yUlcsV3TvYy/pb765U/uOndJla5bpP/3EBr3pkrOUyVja5QEAAElmttndN9Z9jhC2sA2Xyrpvyx79zTd36qXDg7rorC7d9obz9TOvWaX2XJR2eQAALGqEsEWgVK7ogSf26W++uUPP7R9QdyGnX7xijd5x5TptWLkk7fIAAFiUCGGLiLvrOzsP6e7HXtJXn3pZxbLrdef16uYr1+qnLjlLXe25tEsEAGDRmCiEZee6GIRlZrpmwwpds2GF+k8M6/Obd+mex17Sf/6HbcpnM/rxV/TpLa9ZpTdecpaWtHH6AQBICz1hi0Cl4vreriN64Il9evD7+7T/+LDy2YzecOEK/fhFK3XdK/q0treQdpkAALQchiMxqlJxbX7piP71iX362vb92n1kSJJ0QV+nrrtopa67qE9XntertiyT+gEAmC1CGOpyd+3sP6lvPntADz3Xr+/+4LBGShV15CJdc8FybVzfqyvWdes1a7rVkSeUAQAwXcwJQ11mpg0rl2jDyiX69WvP1+BISY++cEjffLZf33r+oL7+zAFJUpQxXbKqS1es69Hl67p1xboerestyIz1yAAAmCl6wtDQ4ZMj2rrriLb88Ki2vHRE23Yd1cmRsiRpWUdOF5/dpUtWLR29fcVZXfSYAQBQg54wzEhvZ14/efFZ+smLz5IklSuu5/af0JaXjuipvcf1zL7j+vymXaPBzEw6b3mnLl7VpYvPPh3OVnd3sIo/AADjEMIwZfGw5FJdsmrpaFul4tp9ZEjbXz6uZ/ad0PZ9x/X03uP68pMvq9rJuqQtq1ectUTn9y3R+uUFrV/RqfXLO7V+RSfLZAAAFi3+AmJWMhnTuuUFrVte0E+/8uzR9pPDJT23/4SeefmEntl3XM+8fELfer5fX9g8PGb/vq42rV9e0LreTq3ubtfqng6d0x1/re7u4NJLAICWRQhDEJ1tWV2+rkeXr+sZ0z44UtKLBwf14qGT+sHBk/rhoZN68eCgHtl5UPuPn1Jl3BTF5Z35OJgtS4JZT0cc1roLOqe7Xb2deT4gAABYkAhhmFOFfFaXnrNUl56z9IzniuWKXj52SnuPDmnP0aHk9pT2HB3Sjv4BPfRcv4aK5TH7tOcyOntpu1YsaVNfV1ud2/zoY3rVAADzCSEM80Yuymhtb6Hh6v3urqODRe2pDWlHhrT/xLAOnhjW8wcG9MjOQzo2VKy7f1d7Vn1L2rSiq019SUjr7cyru5DTso74q7uQV3dyf2lHThEfKAAABEIIw4JhZurpzKunM69XrV7WcLvhUlmHBkZ0cGBY/SeGa25H1H9iWP0Dw9q+77gefm5YJ4ZLE77n0vaslhVy6u6Iw9rSjpy6O3LqTtqWdeSS53NjtqPXDQAwGUIYWk5bNhqd3D+ZkVJFx08VdXSwqGNDIzo2FN8/OljU0aGijg8VdXRwREeHijo2VNSeI0Oj98vjJ7DVyGczWtqe1ZK2rJZUb9ty6hrXNvo4aetqy2lJe1aFfKSOfKRCLlI2yjTz2wMAmCcIYVjU8tmMViyJ54xNh7trYLiUhLfi6duhER0djMPbieGSBk6VNJDc7jk6pIHhogZOlXTiVEmlCULc+BoLSSArtCUBLRfFbW3ZuD0fqSOfVWc1vOXj7QrJ/Y7kfmfN/fZcxHArAKSIEAbMgJmpqz2nrvac1s5gf3fXcKkyGtAGhuNgFt8WNThS1tBIWYMjZQ0WSxocju8PFUtx23BZ/QPDGjw8qKGRsk4OlzRULKtYnt4VMLIZU3suUls2o7ZsRu25SPnktq3ObVsuo/ZsNPZ2dP9I7bn4ti037nE2o3w2o3yUUa56GxmfbAWwqBHCgBSYxeGnPRdNuxduIsVyZTTAnRwpnQ5yI0l4GylrKLl/qljRcKms4VJFp4r1b48Ojmi4VDnjuVPF8hnLicxEPorDWS6yOKRlM8pFmdH208+PfRyHOVM+ipTLmtqieJsoMuUyGWUjUzbKKJeJb7MZi9sy8XtVn4uS53Pjnqu3fTZjyib7EB4BNAMhDGghuSijZR0ZLevIBX+vYjkOZ8PFsk5Vb5NgN/52pFTRSLmikVJFxeR2pOzxbU1bsVzRcPnMtpPDJQ1X28oVFUs++nrV27lUDW1xUBsX+JIgGGXsjFAXZeL7GUsCXWSK7HS4G/+VzZgyo+EvE28bJc9bNUTWvN6YfTOKMjr9ntXXa/AakcXhMsqYMiZlLN4nsuRxsm9kJssoaTdlMjrdbiKgAtMQNISZ2fWSPiYpkvQpd//IuOctef5GSYOS3uPuW0LWBKA5cknv03y49JS7q1RxlSuuYrmiUtlVrMS3pbKrVKmoVPNcqVJRsTx2+zPaKq5SOW6r7h+/XkXFmufKldP71j4Xv87p504VKypXysn2rkqyX7niKrurXE7aq8dSjturxzXRB0Hmk9oAV71fDWhRxsY8d0bwqw16DbYfE/ySXslo3Huebj/zdav7nt4/ed3qe020f82+1XYzkyluN4uP33Q6kGZMSbsl3x8bfWyKt6l9nMnU2V/xrcZs12D/0TCc7Jc5vX9cQvW+jat1CrXFLzrx/mNqIJBPJti/nmYWSfprSW+StFvS42Z2v7s/XbPZDZIuTL5eJ+kTyS0ATJlZ3OuUi9Syy4O4uyqu0UBWqlRUqWhMkCslobDsyTblmlBXqahcu/3o68S3lWQfT96j4tUvJe3V7TT6XLmS3E/ay15nu6QeTx6PPld9T6/znpUGr1XzuFSuJPtr7HaNXrdRjTXb1z5eIJl33msY4mrCYNI8GixrQ9zYgJsE2cy4/ZP3Gb99/P61z9cG2rj9hledrf/44xfM/TcmEfK/sFdK2uHuL0iSmd0r6SZJtSHsJkl3ubtLetTMus1slbvvC1gXACw41R6f059obc2wOV94NcTVBMPRcFc53e5JCHRp9PHpdskVBzofDXbVx2O31/j9a2qo3d91+rUrNe8rn2T/mjrGvtfp7Xx8bWNeo1GttTVNsn/dWuvsX1PHhPvrzPrHf288+Z4nu9Wcq/h+2v9pCxnCVkvaVfN4t87s5aq3zWpJhDAAQGpGh+LEkBrCCbkKZL2f3PEdvFPZRmZ2m5ltMrNN/f39TSkOAAAgTSFD2G5pzBJKayTtncE2cvc73X2ju2/s6+treqEAAABzLWQIe1zShWZ2npnlJd0s6f5x29wv6d0Wu0rSMeaDAQCAxSDYnDB3L5nZ+yV9RfEM0s+4+1Nm9r7k+TskPah4eYodipeouDVUPQAAAPNJ0AV+3P1BxUGrtu2Omvsu6faQNQAAAMxHIYcjAQAA0AAhDAAAIAWEMAAAgBQQwgAAAFJACAMAAEgBIQwAACAFhDAAAIAUEMIAAABSYPF6qQuHmfVL+uEcvNUKSQfn4H3mmw3pPjQAAAcKSURBVMV63NLiPXaOe/FZrMe+WI9bWrzHPh+O+1x3r3vh6wUXwuaKmW1y941p1zHXFutxS4v32DnuxWexHvtiPW5p8R77fD9uhiMBAABSQAgDAABIASGssTvTLiAli/W4pcV77Bz34rNYj32xHre0eI99Xh83c8IAAABSQE8YAABACghh45jZ9Wb2rJntMLPfSbueUMxsrZl9w8y2m9lTZvaBpP2/m9keM9uafN2Ydq0hmNmLZvb95Bg3JW29ZvbvZvZ8ctuTdp3NZGYX1ZzXrWZ23Mw+2Krn3Mw+Y2YHzOzJmraG59jMfjf5vX/WzH46napnr8Fx/5mZPWNmT5jZfWbWnbSvN7OhmnN/R3qVz16DY2/4893i5/wfao75RTPbmrS32jlv9LdsQfyuMxxZw8wiSc9JepOk3ZIel/QOd3861cICMLNVkla5+xYz65K0WdJbJf2SpAF3//NUCwzMzF6UtNHdD9a0/amkw+7+kSSA97j7h9KqMaTkZ32PpNdJulUteM7N7A2SBiTd5e6vStrqnmMzu1TSPZKulHSOpK9JeoW7l1Mqf8YaHPebJf1fdy+Z2Z9IUnLc6yU9UN1uoWtw7P9ddX6+W/2cj3v+o5KOufsfteA5b/S37D1aAL/r9ISNdaWkHe7+gruPSLpX0k0p1xSEu+9z9y3J/ROStktanW5VqbtJ0ueS+59T/Ivcqt4oaae7z8XCx6lw94clHR7X3Ogc3yTpXncfdvcfSNqh+N+DBafecbv7V929lDx8VNKaOS9sDjQ454209DmvMjNT/J/re+a0qDkywd+yBfG7Tggba7WkXTWPd2sRBJPkf0aXS/pu0vT+ZNjiM602JFfDJX3VzDab2W1J21nuvk+Kf7ElrUytuvBu1th/lBfDOZcan+PF9Lv/a5K+XPP4PDP7npk9ZGbXplVUYPV+vhfLOb9W0n53f76mrSXP+bi/ZQvid50QNpbVaWvp8VozWyLpi5I+6O7HJX1C0gWSXitpn6SPplheSK939ysk3SDp9qQ7f1Ews7ykn5P0+aRpsZzziSyK330z+7CkkqS7k6Z9kta5++WSfkvS/zGzpWnVF0ijn+9Fcc4lvUNj/8PVkue8zt+yhpvWaUvtvBPCxtotaW3N4zWS9qZUS3BmllP8Q3u3u/+TJLn7fncvu3tF0v/WAu2en4y7701uD0i6T/Fx7k/mF1TnGRxIr8KgbpC0xd33S4vnnCcaneOW/903s1skvUXSOz2ZDJwMyRxK7m+WtFPSK9Krsvkm+PleDOc8K+kXJP1Dta0Vz3m9v2VaIL/rhLCxHpd0oZmdl/QW3Czp/pRrCiKZJ/BpSdvd/S9q2lfVbPbzkp4cv+9CZ2adyQROmVmnpDcrPs77Jd2SbHaLpC+lU2FwY/5nvBjOeY1G5/h+STebWZuZnSfpQkmPpVBfEGZ2vaQPSfo5dx+sae9LPqQhMztf8XG/kE6VYUzw893S5zzxU5Kecffd1YZWO+eN/pZpofyuuztfNV+SblT8Ccmdkj6cdj0Bj/PHFHfBPiFpa/J1o6S/k/T9pP1+xZ86Sb3eJh/7+ZK2JV9PVc+zpOWSvi7p+eS2N+1aAxx7QdIhSctq2lrynCsOmvskFRX/7/e9E51jSR9Ofu+flXRD2vU3+bh3KJ4HU/1dvyPZ9heT34FtkrZI+tm06w9w7A1/vlv5nCftn5X0vnHbtto5b/S3bEH8rrNEBQAAQAoYjgQAAEgBIQwAACAFhDAAAIAUEMIAAABSQAgDAABIASEMAKbIzK4zswfSrgNAayCEAQAApIAQBqDlmNmvmtljZrbVzD5pZpGZDZjZR81si5l93cz6km1fa2aPJhd4vq96gWcz22BmXzOzbck+FyQvv8TMvmBmz5jZ3cmK3QAwbYQwAC3FzC6R9MuKL9L+WkllSe+U1Kn4mplXSHpI0h8mu9wl6UPu/hrFK6tX2++W9NfufpmkaxSvSC5Jl0v6oKRLFV994fXBDwpAS8qmXQAANNkbJf2IpMeTTqoOxRfvrej0hYz/XtI/mdkySd3u/lDS/jlJn0+uLbra3e+TJHc/JUnJ6z3mybX4zGyrpPWSvh3+sAC0GkIYgFZjkj7n7r87ptHs98dtN9E12yYaYhyuuV8W/44CmCGGIwG0mq9LepuZrZQkM+s1s3MV/3v3tmSbX5H0bXc/JumImV2btL9L0kPuflzSbjN7a/IabWZWmNOjANDy+B8cgJbi7k+b2e9J+qqZZSQVJd0u6aSkV5rZZknHFM8bk6RbJN2RhKwXJN2atL9L0ifN7I+S13j7HB4GgEXA3CfqkQeA1mBmA+6+JO06AKCK4UgAAIAU0BMGAACQAnrCAAAAUkAIAwAASAEhDAAAIAWEMAAAgBQQwgAAAFJACAMAAEjB/wekgO6zFAyCzAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (10, 6))\n",
    "plt.plot(model.loss_curve_)\n",
    "plt.xlabel('epoch')\n",
    "plt.ylabel('loss')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到随着迭代轮数的增加，loss降低地越来越缓慢"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 课后部分：请你在一张图内，绘制出学习率为3，学习率为1，学习率为0.1，学习率为0.01，四个模型的损失函数变化曲线，最大迭代轮数为250轮。并结合参数学习率对图形进行分析\n",
    "\n",
    "提示：分别训练4个模型，然后在一张图中分别绘制4个模型的loss_curve_即可。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
